Emerald FullText Article : Google Scholar revisited

Purpose – The purpose of this paper is to revisit Google Scholar. Design/methodology/approach – This paper discusses the strengths and weaknesses of Google Scholar. Findings – The Google Books project has given a massive and valuable boost to the already rich and diverse content of Google Scholar. The downside of the growth is that significant gaps remain for top ranking journals and serials, and the number of duplicate, triplicate and quadruplicate records for the same source documents (which Google Scholar cannot detect reliably) has increased. Originality/value – This paper discusses the strengths and weaknesses of Google Scholar. Article Type: General review Keyword(s): Data collection; Worldwide web; Document delivery. Journal: Online Information Review Volume: 32 Number: 1 Year: 2008 pp: 102-114 Copyright © Emerald Group Publishing Limited ISSN: 1468-4527 Google Scholar had its debut in November 2004. Although it is still in beta version, it is worthwhile to revisit its pros and cons, as changes have taken place in the past three years both in the content and the software of Google Scholar – for better or worse. Its content has grown significantly [dash ]– courtesy of more academic publishers and database hosts opening their digital vaults to allow the crawlers of Google Scholar to collect data from and index the full-text of millions of articles from academic journal collections and scholarly repositories of preprints and reprints. The Google Books project also has given a massive and valuable boost to the already rich and diverse content of Google Scholar. The downside of the growth is that significant gaps remained for top ranking journals and serials, and the number of duplicate, triplicate and quadruplicate records for the same source documents (which Google Scholar cannot detect reliably) has increased. While the regular Google service does an impressive job with mostly unstructured web pages, the software of Google Scholar keeps doing a very poor job with the highly structured and tagged scholarly documents. It still has serious deficiencies with basic search operations, does not have any sort options (beyond the questionable relevance ranking). It offers filtering features by data elements, which are present only in a very small fraction of the records (such as broad subject categories) and/or are often absent and incorrect in Google Scholar even if they are present correctly in the source items. These include nonexistent author names, which turn out to be section names, subtitles, or any part of the text, including menu option text which has nothing to do with the document or its author. This makes “F. Password” not only the most productive, but also a very highly cited author. Page numbers, the first or second segment of an ISSN, or any other four-digit numbers are often interpreted by Google Scholar as publication years due to “artificial unintelligence”. As a consequence, Google Scholar has a disappointing performance in matching citing and cited items; its hit counts and citation counts remain highly inflated, defying the most basic plausibility concepts when reporting about documents from the 1990s citing papers to be published in 2008, 2009 or even later in the twenty-first century. In spite of the deficiencies and shoddiness of its software the free Google Scholar service is of great help in the resource discovery process and can often lead users to the primary documents in their library in print or digital format and/or to open access versions of papers which otherwise would cost more than $30-$40 each through document delivery services. Google Scholar can act at the minimum as a free, huge and diverse multidisciplinary I/A database or a federated search engine with limited software capabilities, but with the superb bonus of searching incredibly rapidly the full-text of several million source documents. However, using it for bibliometric and scientometric evaluation, comparison and ranking purposes can produce very unscholarly measures and indicators of scholarly productivity and impact. 19/05/09 13:07 Emerald FullText Article : Google Scholar revisited Pagina 2 di 12 http://www.emeraldinsight.com/Insight/ViewContentServlet;jsessioni...Filename=Published/EmeraldFullTextArticle/Articles/2640320108.html Background and literature On the third anniversary of Google Scholar I give a summary of the pros and cons of Google Scholar, focusing on the increasingly valuable content and on the decreasingly satisfactory software features which must befuddle searchers and ought to be addressed by the developers. I discuss here Google Scholar from the perspective of some of the traditional database evaluation criteria that have been used for decades (Jacsó, 1998). I complement this paper with an unusually long bibliography of some of the most relevant English-language articles by competent information professionals. For many of the citations I provide the URL of an open access preprint or reprint version, or of the original version published in an open access journal, to offer readers convenient access to the papers and understand the opinion of the authors. Re-reading these papers in preparation for this review was a great pleasure, even when my opinion did not agree with that of the reviewers. The balance of pro and con arguments and evidentiary materials presented by competent information professionals has been rewarding and has motivated my creation of this bibliography. It does not include references to papers which are dedicated to the citation counts of articles as presented by Google Scholar. These will be provided in follow-up papers which discuss the strengths and weaknesses of using Scopus, Web of Science and Google Scholar to determine the Hirsch-index and derivative indexes for measuring and comparing research output quantitatively. After the launch of Google Scholar it received much attention, just as anything does that relates to Google, Inc. Within the first few months of its debut, there were a number of reviews in open access web columns (Price, 2004; Jacsó, 2004; Goodman, 2004; Gardner and Eng, 2005; Abram, 2005; Tenopir, 2005), and three web blogs were launched dedicated to Google Scholar (Sondemann, 2005; Giustini, 2005), or partially dedicated (Iselid, 2006). These were followed by reviews in traditional publications (Jacsó, 2005a; Myhill, 2005; Notess, 2005, O'Leary, 2005, Giustini and Barsky, 2005; Noruzi, 2005; Adlington and Benda, 2006; Cathcart and Roberts, 2006) focussing on the content and software aspects of Google Scholar. These were well complemented by a number of essays, editorials and surveys pondering the acceptance, use, promotion and “domestication” of Google Scholar as one of the endorsed research tools for students and faculty in academic institutions (Kesselman and Watsen, 2005; Price, 2005; Anderson, 2006; Gorman, 2006; Mullen and Hartman, 2006; Friend, 2006; Hamaker and Spry, 2006; York, 2006; Helms-Park et al., 2007; Schmidt, 2007; Taylor, 2007). As Google Scholar became more intensively used, several research papers started to put it into context by comparing Google Scholar's performance with a single database (Schultz, 2007), federated search engines (Felter, 2005; Giustini and Barsky, 2005; Chen, 2006; Sadeh, 2006; Donlan and Cooke, 2006; Haya et al., 2007; Herrera, 2007), citationenhanced databases such as Web of Science and/or Scopus (Bauer and Bakkalbasi, 2005; Jacsó, 2005b; Jacsó, 2005c; Yang and Meho, 2006; Norris and Oppenheim, 2007), or with a mix of these and traditional scholarly indexing/abstracting databases (White, 2006). There is increasing specialisation in researching Google Scholar, applying the traditional database evaluation criteria such as size, timeliness, source type and especially breadth of journal coverage (Jacsó, 1997) in a consistent manner in the context of a very non-traditional database which piggybacks on other sources rather than creating its own (Wleklinksi, 2005; Vine, 2005; Vine, 2006; Neuhaus et al., 2006; Pomerantz, 2006; White, 2006; Mayr and Walter, 2007; Walters, 2007). The recent incorporation of books in Google Scholar from Google Book Search (which after a poor debut with deficient software features, turned around and introduced within a month far more sophisticated software than Google Scholar in three years), spawned useful research (Hauer, 2006; Lackie, 2006; Goldeman and Connolly, 2007), as did the only good new software feature of Google Scholar which led users to the full-text digital source document in the users' library through Open-URL resolvers (Grogg and Ferguson, 2005; O'Hara, 2007; Lagace and Chisman, 2007). There is one additional research area where Google Scholar will play an important role: its use for bibliometric and scientometric evaluation of the performance of researchers, which is such a complex issue that it deserves to be discussed in a separate paper, with its own rich set of references.